The present invention generally relates to a neural network learning system, and more particularly to a neural network learning system in which a neural network model based on a unified theory using mathematical statistics is constructed and used. By the unified theory, conventional neural networks such as Boltzmann machine and function approximation neural networks (FANN) are generalized, and the disadvantages of the conventional neural networks are eliminated.
Recently, applications of neural networks to pattern recognition, voice recognition, robotic control and other techniques have been studied, and it is recognized that the neural network applications are very useful in those field. In the prior art, a known neural network learning system obtains an input-output relationship by taking given inputs and desired outputs corresponding to the given inputs, so that learning of a neural network is performed in accordance with the input-output relationship.
FIG. 1 shows an input-output relationship which is inferred by a conventional neural network learning system of the type as described above. In the neural network learning system shown in FIG. 1, an input-output relationship is inferred from a set of given input and output samples [(xi, yi); i=1, 2, . . . , N]. A parameter w, which satisfies the function y=.phi.(w, x) indicating the input-output relationship with the maximum likelihood, is obtained. In other words, the output y (=.phi.(w, x)) in accordance with the input-output relationship is obtained from the given teaching data [(xi, yi)] in the conventional neural network learning system.
However, the learning performed by the conventional neural network learning system described above relates to correspondence between one input and one output only. Generally, the known neural networks, such as the Boltzmann machine or the FANN, cannot estimate the variance of outputs, cannot deal with the learning with respect to correspondence between one input and multiple outputs, and cannot judge whether a given input is known or unknown. Also, in the above described learning system, it is impossible to obtain an input for a given output in accordance with the inferred input-output relationship in the reverse manner. Also, it is impossible to estimate the reliability of the output y obtained through the above described inference.